-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
new utility functions #521
Conversation
@gfabbris This is ready for review. |
I don't see how it is possible to get values from baseline with any of the three functions, when only a key fragment is given. Can key_fragment and strict also be implemented for getRunData and getRunDataValue? |
There is a |
The real situation is that the databroker v2 search is slow compared with the v1 interface. For now, rely on the v1 interface. |
Time some test functions to compare: import databroker
catalog_name = "training" # on my test workstation
%%timeit
cat = databroker.catalog[catalog_name]
for idx in range(-20, 0):
run = cat.v1[idx]
uid = run.start["uid"]
# if "baseline" in run.stream_names:
# bl = run.table(stream_name="baseline")
# print(idx, uid)
%%timeit
cat = databroker.catalog[catalog_name]
for idx in range(-20, 0):
run = cat.v2[idx]
uid = run.metadata["start"]["uid"]
# if hasattr(run, "baseline"):
# baseline = run.baseline.read()
# print(idx, uid)
In commented, the baseline data is not actually read from databroker. Roughly twice as long to use the v2 interface instead of the v1 interface. In [9]: databroker.__version__
Out[9]: '1.2.2' The database is on a LAN MongoDB server. |
Needs unit testing and could benefit from a lower-case match instead of explicit case match. |
@gfabbris , @strempfer , @CrI3 ; This is ready for review. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great.
In getStreamValues db could be moved after key_fragment like it is done for listRunKeys. Otherwise there is an error if no db is given: getStreamValues(-1,'lake') does not work, but getStreamValues(-1,db,'lake') does.
@strempfer Great catch. Turns out there are no unit tests yet for |
@strempfer : Added unit tests. If you do not specify a |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Just one question that can be addressed in the future.
_db = db | ||
|
||
if len(query) != 0: | ||
_db = _db.v2.search(query) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a way combine these searches? I wonder if that would be faster.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good
Three new utility functions from APS 4-ID polartools will help users to find data from databroker runs. Fixes #502
Thanks @gfabbris & @strempfer!